Autonomous vehicle fleet model training and testing

ABSTRACT

A method and system of using excess computational resources on autonomous vehicles. Such excess computational resources may be available during periods of low demand, or other periods of idleness (e.g., parking). Where portions of computing resources are available amongst a fleet of vehicles, such excess computing resources may be pooled as a single resource. The excess computational resources may be used, for example, to train and/or test machine-learning models. Performance metrics of such models may be determined using hardware and software on the autonomous vehicle, for example sensors. Models having performance metrics outperforming current models may be considered as validated models. Validated models may be transmitted to a remote computing system for dissemination to a fleet of vehicles.

RELATED APPLICATIONS

This application claims priority to U.S. patent application Ser. No.15/803,603, filed Nov. 3, 2017, which is incorporated herein byreference

BACKGROUND

Autonomous vehicles that receive limited or no input from a passengermay include computing systems engineered to operate under normativeconditions and, to the extent extraneous circumstances the autonomousvehicle may encounter are contemplated, some unusual conditions.However, these systems may not be sufficient (e.g. may not have properlytrained models or adequate amount of computing power) to handlecontingencies encountered during operation of the autonomous vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentify the figure in which the reference number first appears. Thesame reference numbers in different figures indicate similar oridentical items.

FIG. 1 illustrates a pictorial flow diagram of an example process fortraining and/or testing machine-learning (“ML”) models at a vehicle.

FIG. 2 illustrates a block diagram of an example architecture forcontrolling operation of a vehicle for training and/or testing MLmodels.

FIG. 3 illustrates a block diagram of an example vehicle controller andan example experimental vehicle controller.

FIG. 4A illustrates an example testing and/or training scenarioinvolving multiple vehicles.

FIG. 4B illustrates an example evaluation coordination between twovehicles for testing and/or training an ML model.

FIG. 5 illustrates a flow diagram of an example process for controllinga vehicle to conduct training and/or testing.

FIG. 6 illustrates a flow diagram of an example process for testingand/or training an ML model at a vehicle.

FIG. 7 illustrates a flow diagram of an example process for diverting avehicle to conduct an evaluation to train and/or test an ML model.

DETAILED DESCRIPTION

This disclosure is generally directed to techniques (e.g., machines,processes) for using unused computational resources, such as processorclock cycles for processing data, at one or more autonomous vehicles.The autonomous vehicles may be configured to use unused computationalresources to train and/or test machine-learning models and/or to poolcomputational resources to process high-volume computational tasks. Forexample, autonomous vehicles may include one or more machine-learning(ML) models to operate the vehicle (e.g., models for identifying and/ortracking objects; route-planning; trajectory-planning; evaluatingdecisions; etc.). An autonomous vehicle may require updates and/oradditions to its ML models to handle previously unencountered scenariosor to improve one or more driving parameters (e.g., an ability tocorrectly predict and/or classify an object, vehicle efficiency,comfort, etc.). For example, an additional and/or revised ML model maybe transmitted to the autonomous vehicle to handle sensor noise causedby the position of the sun at a particular intersection and time of dayso as to provide higher accuracy in predicting object classifications.

The techniques discussed herein include using unused computational powerof an autonomous vehicle to train and/or test (e.g., an ML model). Inthe interest of prioritizing safety and unhindered operation, ratherthan merely relying on synthetic simulation and testing of the ML modeloffline (i.e., running the ML model without using the hardware of avehicle, live sensor data of a vehicle, and/or without operating avehicle), the techniques discussed herein include testing and/ortraining an ML model online at an autonomous vehicle (i.e., using thehardware and/or live sensor data of the autonomous vehicle and/oroperating the autonomous vehicle using the ML model). This may includeusing unused computational power at an autonomous vehicle, using sensordata captured by sensors of the autonomous vehicle, and/or using the MLmodel in a pipeline to operate the autonomous vehicle.

In some examples, this may include detecting that one or more autonomousvehicles are or are predicted to be available to conduct testing and/ortraining and diverting the one or more autonomous vehicles from“in-service” operation (e.g., providing ride services, conducting regionmapping, etc.) to conduct the testing and/or training. However, in someexamples, one or more of the autonomous vehicles may conduct the testingand/or training while remaining in an “in-service” operation state. Forexample, an autonomous vehicle may be operating in a computationallylight scenario that requires less cycles of a processor of theautonomous vehicle to be allocated to nominative operation of thevehicle (e.g., the vehicle is driving down a long, straight roadway withno intersections and low or no traffic, the vehicle is at a stop light,the vehicle is parked).

An autonomous vehicle and/or a remote computing device (e.g., ateleoperations device, a fleet management device, another autonomousvehicle) may detect that the autonomous vehicle is or will be availableto conduct testing and/or training. This may include determining aprocessing bandwidth at one or more processors of a vehicle (e.g.,determining a number of threads allocated to the one or more processors,a number of cycles devoted to a thread, a priority of an allocatedthread or a thread-to-be-allocated, and/or a remaining number of threadsthat may be allocated to the one or more processors, etc.), determininga likely processing load for a current route of a vehicle, and/ordetermining availability of fleet vehicles from historical data (e.g.,service demand data). In some examples, upon determining that autonomousvehicle(s) are available to conduct testing and/or training, a remotecomputing device may transmit a test mission to the autonomousvehicle(s) that includes a time and/or time period, a location, acondition, and/or instructions for at least one maneuver of theautonomous vehicle(s) for operation during testing and/or training.

In some examples, an autonomous vehicle may include a vehicle controllerthat includes one or more machine-learning models. ML models mayinclude, for example, one or more of an artificial neural network(“ANN”), such as a convolutional neural network (“CNN”); a directedacyclic graph (“DAG”) (e.g., where the nodes are organized as a Bayesiannetwork); a support vector machine (“SVM”), etc. In some examples, MLmodels of the vehicle controller may be configured to perform variousoperations such as, for example, localizing the autonomous vehicle,detecting objects, classifying objects, tracking objects, generatingtrajectories, selecting one or more trajectories for controlling theautonomous vehicle, determining whether to communicate data with ateleoperations device, etc.

In some examples, testing and/or training may include further trainingan existing (“target”) ML model, which may include, for example, editingthe target ML model based on processing additional iterations of data tochange weights of connections in a neural network; and/or testing anexperimental ML model at an autonomous vehicle (e.g., running theexperimental ML model on the autonomous vehicle to verify that theexperimental ML model works as intended and/or whether the experimentalML model meets or exceeds performance of prior configurations of MLmodels). During normative operation of the autonomous vehicle, when theautonomous vehicle is not conducting testing and/or training, thevehicle controller may be configured with normative ML models. Duringtesting and/or training, the autonomous vehicle may instantiate a secondvehicle controller and/or a portion of the first vehicle controller thatadditionally includes an experimental ML model or replaces a normativeML model with the experimental ML model. Additionally, or alternatively,if the vehicle controller is training a target ML model, the autonomousvehicle may instantiate a second vehicle controller and/or a portion ofthe first vehicle controller that includes the target ML model.

In some examples, this may include running at least part of a secondcontroller, with the experimental ML model and/or target model, on avirtual machine, container, and/or running at least part of the secondcontroller across a single system image that comprises processing and/ormemory components of multiple devices (e.g., multiple autonomousvehicles, an autonomous vehicle and a remote computing device).

In examples employing a virtual machine or container, the autonomousvehicle may record a machine state (e.g., a hypervisor state, anoperating system state, a container manager state, a runtime systemstate, and/or an application state) so that the machine state may besaved for continuance of the testing and/or training at a later point intime if the testing and/or training is interrupted and/or migrated to asecond autonomous vehicle and/or remote computing device. In someexamples, the autonomous vehicle may simultaneously conduct testingand/or training during “in-service” operation by running at least partof the vehicle controller (e.g., an experimental ML model, a target MLmodel that is being trained) on a virtual machine or in a separatecontainer (i.e., using a separate runtime system from the runtime systemof the first controller) while the vehicle controller is run on a nativeoperating system, a second virtual machine, or a different container.

In some examples, multiple autonomous vehicles may be involved inrunning one testing and/or training. The autonomous vehicles may createan ad hoc network connection between the multiple vehicles to coordinatemaneuvers, positioning, and/or sensor data recording among the vehicles;and/or to split data processing amongst processing devices of themultiple vehicles (e.g., via a single system image).

In some examples, one autonomous vehicle and/or a remote computingdevice may instantiate a distributed computing architecture such as, forexample, a single system image, over multiple autonomous vehicles toshare available resources among the vehicles such as, for example,sensor data, processing availability, storage availability. In someexamples, data-to-be-processed may be distributed to processors ofdifferent vehicles according to the distributed computing architecture.In some examples, the distributed computing architecture may be used totest and/or train ML models, including analyzing performance of one ormore vehicles that are operating using an ML model. For example, a firstvehicle may operate using an experimental ML model while a secondvehicle monitors the first vehicle. In such examples, the second vehiclemay monitor sensor data captured by the first vehicle and/or the secondvehicle to analyze performance of the first vehicle. As non-limitingexamples, such analysis may include, for example, how well the firstvehicle adheres to driving rules of the road, an amount of energyconsumed to traverse a road segment, an accuracy of object segmentationand/or classification, etc. In those examples where the second vehiclemonitors the first, processing power of the first vehicle may be devotedto operating using the experiment ML model, so as not to impactprocessing performance of the first vehicle. This is discussed furtherherein as “determining performance metrics.”

In an additional or alternate example, a remote computing system and/oran autonomous vehicle may detect a “high processing load” condition on acomputing system of an autonomous vehicle and may transmit an indicationof the “high processing load” condition to one or more other autonomousvehicles. The high processing load condition may be a condition thatcauses the autonomous vehicle performance to degrade because therequired time to process data caused by the condition meets a threshold.For example, the condition might be an anomalous environment (e.g.,enormous number of erratic objects) and/or running testing and/ortraining, either of which requiring processing a large amount of data.In that case, a remote computing device and/or an autonomous vehicle maytransmit an indication to one or more other autonomous vehicles toinstantiate a mobile distributed computing resource comprising the oneor more other autonomous vehicles, connected via an ad hoc network andconfigured as a single system image. As such, the performance of thevehicle having a “high processing load” condition may be improved byaccessing processing bandwidth of the other autonomous vehicles (e.g.,running a process and/or thread at another vehicle's processor andreceiving the results of the execution of the process and/or thread fromthe other vehicle). For example, the vehicle may operate using atrajectory, object classification, predicted object path, etc.determined by one or more other vehicles. In an additional or alternateexample, the vehicle may conduct testing and/or training in concert withone or more other vehicles.

In some examples, an autonomous vehicle and/or a remote computing devicemay analyze performance of the autonomous vehicle as it operated using atrained target model and/or an experimental ML model to determinewhether performance of the autonomous vehicle was satisfactory orimproved. Historic drive data of the autonomous vehicle and/or otherautonomous vehicles may be used as a control to which to compare and/orthe test mission may include operating the autonomous vehicle using anormative ML model to establish a control and, separately, using atrained target ML model (i.e., a normative model that has been modifiedby additional iterations of training) and/or experimental ML model.Metrics for experimentally making such a determination may include atemporal logic output based on the vehicle controller output (i.e., howthe autonomous vehicle was driven, including, but not limited to,maximum longitudinal and lateral accelerations, evaluated collisionswith obstacles, compliance with rules of the road, etc.), confidencelevels, an amount of energy required for performing driving maneuvers, anumber of teleoperations interventions, processor load of an ML model,speed of achieving a result by an ML model, etc.

In some examples, if analysis of a trained target ML model and/orexperimental ML model meets a threshold value (e.g., a thresholdtemporal logic score, an average confidence level, a number ofteleoperation interventions, etc.), the model may be transmitted as a“validated ML model” to one or more autonomous vehicles of a fleet. Insome examples, if an autonomous vehicle is able to validate an ML model,the autonomous vehicle may replace a normative ML model with the trainedtarget ML model and/or experimental ML model. In some examples, theautonomous vehicle may transmit the validated ML to a remote computingdevice. Additionally, or alternatively, the autonomous vehicle maytransmit any partial or complete testing and/or training data to anotherautonomous vehicle and/or a remote computing device such as, forexample, sensor data, machine state (e.g., virtual machine state,container state), analysis data, an unvalidated ML model, apartially-trained ML model, etc.

In some examples, where the test mission corresponding to anow-validated ML model includes a location and/or condition, a remotecomputing device may transmit the validated ML model to an autonomousvehicle when it is determined that the autonomous vehicle is likely toencounter the location and/or condition. Therefore, in some examples, anautonomous vehicle operating at one location and/or under one conditionmay be configured with an ML model that has a different structure (e.g.,different node connections, different type of ML model, differentlytrained ML model) than an ML model having the same functionality (e.g.,also an object classification ML model) of the same or a differentautonomous vehicle operating at a different location and/or under adifferent condition.

In some examples, an autonomous vehicle may be configured to switchbetween a first ML model associated with a first location and/or firstcondition and a second ML model associated with a second location and/ora second condition based on detecting that the autonomous vehicle islikely to encounter the second location and/or the second condition. Insuch examples, the autonomous vehicle may have an optimized ML model perlocation, condition, etc. For example, a specialized ML model may beassociated with a geophysical location and/or a condition (e.g., time,weather condition, vehicle pose, sensor availability). As a non-limitingexample, a first ML model may outperform a second ML model in a cityenvironment. In such an example, the autonomous vehicle may use thefirst ML model when driving in a city and the second ML model elsewhere.More generally, an autonomous vehicle and/or a remote computing devicemay determine that the vehicle is or is likely to be at a location(e.g., a route of the autonomous vehicle passes through or near thelocation) and/or meet a condition (e.g., a forecast calls for rain atthe vehicle's location in 15 minutes, the time of day matches thecondition associated with the specialized ML model). The remotecomputing device may transmit the specialized ML model associated withthe location and/or condition to the vehicle based on thisdetermination. In additional or alternate examples, the vehicle mayrequest the specialized ML model or, if the specialized ML model isalready stored at the vehicle, the vehicle may add the specialized MLmodel to a pipeline and/or swap the specialized ML model for acorresponding normative ML model that is not associated with thelocation and/or condition (e.g., a normative ML model accomplishes thesame or similar function as the specialized ML model but the normativeML model not weighted and/or structured in a same manner as thespecialized ML model).

FIG. 1 illustrates a pictorial flow diagram of an example process 100for conducting testing and/or training of an ML model at an autonomousvehicle. At operation 102, the example process 100 may includedetermining to conduct testing and/or training at a first vehicle104(1). In some examples, the vehicles 104(1)-104(3) may include anautonomous vehicle such as, for example, an autonomous vehicleconfigured to operate according to a Level 5 classification issued bythe U.S. National Highway Traffic Safety Administration, which describesa vehicle capable of performing all safety-critical functions for theentire trip, with the driver (or occupant) not being expected to controlthe vehicle at any time. In some examples, determining to conducttesting and/or training at the first vehicle 104(1) may be based atleast in part on a determination that the first vehicle 104(1) is idle,that the first vehicle 104(1) has unused processing capabilities, and/orthat a system load 102(A) does not necessitate the vehicle's operationto meet system performance constraints.

For example, the first vehicle 104(1) and/or a remote computing device106 (e.g., a teleoperations device, a fleet management device, anothervehicle) may determine that a position of the first vehicle 104(1) hasnot changed for a set amount of time, has not received a task request ina set amount of time, and/or has an idle processing unit and/or is, onaverage, only using less than a threshold percentage of processingcapacity. Additionally or alternatively, the first vehicle 104(1) may beselected to conduct testing and/or training based at least in part on acurrent or anticipated location and/or a current or anticipatedcondition at the first vehicle 104(1) that matched a location and/orcondition associated with a target ML model and/or an experimental MLmodel. For example, a particular location and/or condition may cause atarget ML model's output to be unreliable. A remote computing device 106may, in that case, associate that location and/or condition with thetarget ML model and, once a vehicle of a fleet encounters or is about toencounter the location and/or condition, the remote computing device 106may transmit instructions to train the target model and/or test anexperimental model at the location/during the condition.

In an additional or alternate example, the first vehicle 104(1) and/or aremote computing device 106 may determine that a current and/orpredicted demand for the vehicle is unlikely to meet a threshold 108.The measured and/or predicted demand and corresponding threshold mayinclude a number of task requests per time period (e.g., ride requestsper hour, task requests per half-hour), a ratio of vehicles to receivedrequests, an average time before a vehicle appears at a requestedlocation (e.g., average of 10 minutes from the time a ride request issubmitted before an autonomous vehicle appears at a ride requestlocation), etc. and may be measured across a fleet of vehicles, forvehicles of a specified region, and/or for an individual vehicle. Forexample, system load 102(A) may depict a predicted or historical numberof ride requests per hour over a 24-hour period. Note that “demand” and“service performance” are used interchangeably herein to refer to datarelated to demand for vehicles (e.g., number of requests per hour)and/or performance of a fleet of vehicles in response to the demand(e.g., average vehicle arrival time, ratio of vehicles “in-service” tonumber of requests and/or traffic flow density).

This determination may additionally or alternatively include determiningan estimated time period for which the vehicle is available to conductthe testing and/or training and/or a number of vehicles that areavailable for conducting testing and/or training without adverselyaffecting service performance of the fleet (e.g., without causing anestimated arrival time to meet or exceed a threshold, without causing aratio of “in-service” vehicles to traffic flow density to meet or exceeda threshold).

In the example system load 102(A), the y-axis may represent a number ofhistorical vehicle requests on a same day of the week, adjusted forknown events; or it may represent an estimated vehicle arrival time. Ineither example, the first vehicle 104(1) may be determined to beavailable from 8 PM until midnight based at least in part on adetermination that demand and/or service performance is not predicted tomeet the threshold 108 from 8 PM until midnight by having vehicle 104(1)perform additional computations, testing, and/or training. In anotherexample, a remote computing device 106 may determine to transmitinstructions to three vehicles to transition from an “in-service” stateto an “out-of-service” state from 8 PM to 9 PM based at least in part ona determination that doing so is not predicted to increase averagearrival time to a point where it meets a threshold average arrival time.In some examples, one or more autonomous vehicles may conduct testingand/or training during an “in-service” state instead of or in additionto conducting the testing and/or training in an “out-of-service” state.

For example, one or more autonomous vehicles may detect unused (i.e.,unallocated, excess) computational resources at the one or moreautonomous vehicles and may use unused computational resources to testand/or train. In some examples, an autonomous vehicle may detect that atleast one processor of the autonomous vehicle is capable of processingadditional threads. For example, the autonomous vehicle may determinethat in addition to the threads being processed for nominative operationof the autonomous vehicle, the autonomous vehicle is capable ofallocating processing time to additional threads for the testing and/ortraining without reducing allocation of threads for the nominativeoperation of the autonomous vehicle (e.g., by unused clock cycles of theprocessor, by increasing a processor speed when the processor is not yetat a maximum processor speed, by using an idle core of a processor,etc.). These unused computational resources may be available across oneor more processors, memories, and/or one or more vehicles. Moreover,allocation and/or processing of the threads may be conducted over one ormore processors and/or one or more vehicles. This means that, in someexamples, the techniques include detecting that at least one processorof multiple autonomous vehicles can handle additional threads andassigning allocation and processing of particular threads to differentprocessors of different autonomous vehicles. In some examples, anautonomous vehicle may also predict that processing power of a processorof the vehicle is likely to be unused (and may thereby be allocatedthreads of a testing and/or training application) based at least in parton route information (e.g., an upcoming long straight roadway withoutintersections and low traffic,), historical data (e.g., processing powerassociated with anticipated maneuvers along a route), etc.

In some examples, the first vehicle 104(1) and/or a remote computingdevice 106 may determine to conduct testing and/or training (or othercomputation) at the first vehicle 104(1) based at least in part onvehicle confidence levels 102(B) and determining a correlation between adegradation 110 in vehicle performance and a condition and/or location.For example, vehicle confidence levels 102(B) are depicted as a graph ofvehicle confidence levels over time. Autonomous vehicles may calculate aconfidence level based at least in part on data available to the vehiclecontroller such as, for example, sensor data, data from a remotecomputing device (e.g., weather conditions, traffic conditions, safetyalerts), and/or data produced or stored by components of the vehiclecontroller (e.g., object classifications, object tracks, maps ofavailable driving surfaces, potential trajectories, route). In someexamples, these confidence levels may be expressed as a percentage(e.g., the autonomous vehicle is 97% confident that a control signalproduced by the vehicle controller to control a drive system of thevehicle will result in safe operation of the vehicle).

Element 110 indicates a degradation in a confidence level of thevehicle. In some examples, the first vehicle 104(1) and/or a remotecomputing device 106 may determine a correlation between the degradation110 and a condition and/or location, based at least in part on currentand/or historical sensor data and/or other data available to the firstvehicle 104(1) and/or the remote computing device 106 (e.g.,municipality alerts, weather conditions, data from other vehicles,traffic information). In some examples, the first vehicle 104(1) and/ora remote computing device 106 may determine to conduct the testingand/or training based on the correlation. In some examples, the testingand/or training may include conducting the testing and/or training whilethe condition is present and/or at the location correlated with thedegradation. The condition may be any one or more of factors such asorientation of the vehicle, ML model output (e.g., a particular objectclassification), weather conditions (e.g., sunny, foggy), surroundingarea conditions (e.g., high number of erratic objects such as smallwildlife, slope of roadway, existence of blockages to view), etc.

In some examples, the vehicle 104(1) may perform additional computationsand/or testing and/or training when parked, or otherwise stopped (e.g.while waiting on a ride request, while at a stoplight, or otherwise). Insuch examples, the vehicle 104(1) may perform these computations and/ortesting and/or training until a request for a ride, or any computationaldemand, is created.

In some examples, upon determining that a vehicle is available toconduct testing and/or training, a remote computing device 106 maytransmit instructions for a test mission to the vehicle that include atime and/or time period, a location, a condition, and/or instructionsfor at least one maneuver of the autonomous vehicle(s) for operationduring the testing and/or training. In some examples, the time and/ortime period configures the autonomous vehicle(s) to transition from an“in-service” state to an “out-of-service” state at or near a timeindicated in the and remain in an “out-of-service” state until receptionof further instructions from the remote computing device, measureddemand increases to a sufficient degree that the threshold is met, thetesting and/or training is completed, the time period expires, and/orthe testing and/or training is frustrated (e.g., autonomous vehicleincapacity, experimental module failure). The location may includeinstructions to begin the testing and/or training at a specifiedgeo-physical location and the condition may include instructions tobegin the testing and/or training once a condition is fulfilled (e.g.,once service demand reduces to meet a threshold, once traffic reaches aspecified density, once the sun is at a specified angle, etc.). In someexamples, the condition may be a condition that causes degradation ofperformance of an autonomous vehicle. For example, a test mission mayspecify a location, an orientation of the autonomous vehicle, weathercondition, and a time corresponding to a particular angle of the sunthat causes shimmer from a body of water to interfere with a perceptionengine of the vehicle controller. The test mission may thereby configurethe autonomous vehicle to conduct the testing and/or training at thetime, location, weather condition (e.g., sunny), and/or orientation. Insome examples, the remote computing device 106 may also provide aninstruction for the first vehicle 104(1) to train an existing ML model(“target” ML model) or test an experimental ML model. The remotecomputing device 106 may also transmit an ML model with the test missionsuch as an experimental ML model.

At operation 112, the example process 100 may include conducting testingand/or training at a first vehicle 104(1). In some examples, the testingand/or training may include further training a target ML model 114and/or testing an experimental ML model 116 In some examples, this mayinclude training a network using data from a data store (e.g.,accessible via a network, streamed to the vehicle from a remotecomputing device, stored at the vehicle) and/or adapting the target MLmodel to newly acquired data (e.g., re-weighting connections of anetwork). In some examples, the testing and/or training may additionallyor alternatively include coordinating (112(B)) multiple vehicles104(1)-104(3).

114116116116When the first vehicle 104(1) is not conducting testingand/or training, the vehicle controller of the first vehicle 104(1) maybe configured with a set of normative ML models 118. During testingand/or training, the first vehicle 104(1) may add an experimental model116 to the normative ML models 118 and/or replace one or more normativeML models 118 with an experimental ML model 116. In some examples, thefirst vehicle 104(1) may additionally or alternatively instantiate asecond vehicle controller that runs the normative ML models 118 andadditionally includes an experimental model 116 or replaces a normativeML model 118 with the experimental ML model 116. Additionally oralternatively, if the vehicle controller is training a target ML model114, the first vehicle 104(1) may instantiate a second vehiclecontroller that includes the target ML model 114 so that it may bemodified without affecting the normative vehicle controller or modifythe target ML model within the first vehicle controller.

In some examples, instantiating the second vehicle controller mayinclude running at least part of the second controller on a virtualmachine, a separate container, and/or running at least part of thesecond controller across a single system image that comprises processingand/or memory components of multiple devices (e.g., multiple vehicles, avehicle and a remote computing device).

In some examples, conducting the testing and/or training may includecoordinating (112(B)) operations of multiple vehicles 104(1)-104(3). Asdiscussed in more detail below, a remote computing device 106 maytransmit instructions to multiple vehicles 104(1)-104(3) configuring themultiple vehicles 104(1)-104(3) to coordinate their operations fortesting and/or training. In some examples, this may include coordinatingmaneuvers, data processing, sensor data capture, network connections,etc. For example, the multiple vehicles 104(1)-104(3) may share databetween the vehicles (e.g., sensor data obtained at a first vehicle istransmitted by that vehicle to one or more other vehicles of themultiple vehicles 104(1)-104(3)), the multiple vehicles 104(1)-104(3)may share processing loads between the vehicles (e.g., a processing taskgenerated at one vehicle may be processed by a processor of a differentvehicle), movements of the multiple vehicles 104(1)-104(3) may beresponsive to instructions sent by one or more of the multiple vehicles104(1)-104(3), etc.

In some examples, conducting testing may include analyzing performanceof the first vehicle 104(1) (and/or multiple vehicles 104(1)-104(3)) asit conducts the evaluation, relative to normative and/or experimentalcontrol performance. For example, analyzing performance may includecomparing data associated with controlling the vehicle according toexperimental control maneuver(s) using normative ML models and/orhistoric maneuver(s) using normative ML models to data associated withcontrolling the vehicle according to the same or similar maneuvers usingan experimental ML model. Analyzing the performance may include findingthat an experimental ML model improves, maintains, or degradesperformance of the maneuvers by the vehicle. As non-limiting examples,performance may include an adherence to rules of the road, an amount ofenergy consumed to perform one or more maneuvers (i.e. an energyefficiency), a ride comfort, and ability to accurately segment and/orclassify objects, etc. In some examples, conducting the testing mayinclude capturing experimental control data by conducting controlmaneuver(s) specified in a test mission while the first vehicle 104(1)is controlled by the vehicle controller configured with normative MLmodels 118. In some examples, the experimental control data may includehistoric drive data of the autonomous vehicle and/or other autonomousvehicles that are controlled using normative ML models 118.

In some examples, the first vehicle 104(1), another (second) vehicle104(2), and/or a remote computing device 106 may determine whethertrained target ML model 114 and/or experimental ML model 116 improvesoperation of the first vehicle 104(1) based at least in part on temporallogic configured to score characteristics of the operation of the firstvehicle 104(1) by the vehicle controller (e.g., penalizing hardaccelerations, rewarding conformity to driving laws), confidence levelsof the vehicle controller during the testing and/or training, processorload of an ML model, speed of achieving a result by an ML model, anumber of teleoperation interventions, an amount of energy consumption,etc, or some weighted combination thereof. These scores, levels, and/ornumbers may be compared to corresponding control data and/or historicdata for a same or similar operation to determine whether an ML modelimproved operation of the first vehicle 104(1).

At operation 120, the process 100 may include transmitting testingand/or training results from the first vehicle 104(1) (or one or morevehicles of multiple vehicles assigned to the testing and/or training)(120(A)) to a remote computing device 106 via a first network 122 and/or(120(B)) to a second vehicle 104(2) via a second network 124. In someexamples, the testing and/or training results may include a trainedtarget model 114 (i.e., a model that formerly existed at the firstvehicle 104(1) but that has had its connections, weights, etc. modifieddue to training), an experimental model 116, and/or performance analysisdata (e.g., data recorded to determine whether the target/experimentalML model improved operation of the vehicle or not). In some examples,the results may include partial data, if, for example, the testingand/or training was paused or interrupted.

In some examples, the first vehicle 104(1) may transmit a trained targetML model and/or experimental ML model to a second vehicle 104(2) for thesecond vehicle 104(2) to conduct testing and/or training using the modeland/or to complete testing and/or training started by the first vehicle104(1). In such examples, transferring the model may comprisetransferring not only the architecture of the model, but also thecurrent parameters of the model (e.g. weights between nodes in a neuralnetwork), number of iterations performed, batch sizes, an indication ofthe data used, etc. In some examples, if performance analysis of atrained target ML model and/or experimental ML model indicates that themodel meets a threshold requirement (e.g., improvement in temporal logicscore over control data, improvement in confidence levels over controldata, lower processor load, quicker to achieve, lower energyconsumption, no or fewer teleoperator interventions compared to the samemaneuver using control data, location, and/or condition) (i.e., thefirst vehicle 104(1) “validates” the model) then the first vehicle104(1) and/or the remote computing device 106 may transmit the model toa second vehicle 104(2). In some examples, the model may be transmittedto an entire fleet of vehicles and/or a portion thereof. Thetransmission may include an indication that the transmitted model isvalidated (i.e., previously tested and verified to improve or maintainperformance of a vehicle for conducting a maneuver) or that the model isin “beta” and requires further testing by the vehicles to which themodel is transmitted. In the latter example, the vehicles to which the“beta” model is transmitted may conduct further training and/or testingon the model and, in some examples, may place the “beta” model in thenormative ML pipeline of the vehicle and, in other examples, may notplace the “beta” model in the normative ML pipeline. In other examples,this may allow for staged rollouts of updated models, for example to asingle “beta” vehicle, then to five vehicles, and so on, until there isenough confidence that the model should be put in the normative MLpipeline.

FIG. 2 is a block diagram of an example architecture 200 including anexample vehicle system 202 for controlling operation of at least onevehicle, such as an autonomous vehicle. In some examples, the vehiclesystem 202 may represent at least a portion of the first vehicle 104(1)and/or any of the multiple vehicles 104(1)-104(3). In some examples,this architecture may be used to control a fleet of vehicles.

In some examples, the vehicle system 202 may include processor(s) 204and/or memory 206. These elements are illustrated in combination in FIG.2, although it is understood that they may be separate elements of thevehicle system 202, and that components of the system may be implementedas hardware and/or software, in some examples. For example, a portion ofthe vehicle controller 226 may be implemented as a hardware processorand another portion may include instructions stored in memory that, whenexecuted by the processor, configure the processor to conductoperations.

Processor(s) 204 may include a uniprocessor system including oneprocessor, or a multiprocessor system including several processors(e.g., two, four, eight, or another suitable number). The processor(s)204 may be any suitable processor capable of executing instructions. Forexample, in various implementations, the processor(s) may begeneral-purpose or embedded processors implementing any of a variety ofinstruction set architectures (ISAs), such as the ×86, PowerPC, SPARC,or MIPS ISAs, or any other suitable ISA. In multiprocessor systems, eachprocessor 204 may commonly, but not necessarily, implement the same ISA.In some examples, the processor(s) 204 may include a central processingunit (CPU), a graphics processing unit (GPU), or a combination thereof.

The example vehicle system 202 may include memory 206. In some examples,the memory 206 may include a non-transitory computer readable mediaconfigured to store executable instructions/modules, data, and/or dataitems accessible by the processor(s) 204. In various implementations,the non-transitory computer readable media may be implemented using anysuitable memory technology, such as static random access memory (SRAM),synchronous dynamic RAM (SDRAM), nonvolatile/Flash-type memory, or anyother type of memory. In the illustrated example, program instructionsand data implementing desired operations, such as those described above,are shown stored within the non-transitory computer readable memory. Inother implementations, program instructions, and/or data may bereceived, sent, or stored on different types of computer-accessiblemedia, such as non-transitory computer readable media, or on similarmedia separate from the non-transitory computer readable media.Generally speaking, a non-transitory, computer readable memory mayinclude storage media or memory media, such as flash memory (e.g., solidstate memory), magnetic or optical media (e.g., a disk) coupled to theexample vehicle system 202 via an input/output (“I/O”) interface 208.Program instructions and data stored via a non-transitory computerreadable medium may be transmitted by transmission media or signals suchas electrical, electromagnetic, or digital signals, which may beconveyed via a communication medium such as a network and/or a wirelesslink, such as may be implemented via a network interface 210.

Furthermore, though illustrated as a single unit in FIG. 2, it isunderstood that the processor(s) 204 and memory 206 may be distributedamong multiple computing devices of the vehicle.

In some examples, the input/output (“I/O”) interface 208 may beconfigured to coordinate I/O traffic between the processor(s) 204, thememory 206, the network interface 210, sensor(s) 212, I/O devices 214,and/or drive system 216. In some examples, the I/O devices 214 mayinclude external and/or internal speaker(s), display(s), passenger inputdevice(s), etc. In some examples, the I/O interface 208 may performprotocol, timing, or other data transformations to convert data signalsfrom one component (e.g., the non-transitory computer readable media)into a format suitable for use by another component (e.g.,processor(s)). In some examples, the I/O interface 208 may includesupport for devices attached through various types of peripheral buses,such as the Peripheral Component Interconnect (PCI) bus standard, theUniversal Serial Bus (USB) standard, or a variant thereof, for example.In some implementations, the function of the I/O interface 208 may besplit into two or more separate components, such as a north bridge and asouth bridge, for example. Also, in some examples, some or all of thefunctionality of the I/O interface 208, such as an interface to thememory 206, may be incorporated directly into the processor(s) 204and/or one or more other components of the vehicle system 202.

The example vehicle system 202 may include a network interface 210configured to establish a communication link (i.e., “network”) betweenthe vehicle system 202 and one or more other devices. For example, thenetwork interface 210 may be configured to allow data to be exchangedbetween the vehicle system 202 and another vehicle 218 (e.g., vehicle(s)104(2) and (3), for example) via a first network 220, and/or between thevehicle system 202 and a remote computing system 222 via a secondnetwork 224. For example, the network interface 210 may enable wirelesscommunication between another vehicle 218 and/or the remote computingdevice 222. In various implementations, the network interface 210 maysupport communication via wireless general data networks, such as aWi-Fi network, and/or telecommunications networks, such as, for example,cellular communication networks, satellite networks, and the like.

The example vehicle system 202 may include sensor(s) 212, for example,configured to sense movement of the example vehicle system 202 throughits environment, sense environmental data (e.g., ambient temperature,pressure, and humidity), and/or sense conditions of an interior of theexample vehicle system 202 (e.g., passenger count, interior temperature,noise level). In some examples, the sensor(s) 212 may include a sensorto monitor operations of the vehicle system 202 to perform analysis ofperformance of the vehicle during normative and/or testing and/ortraining maneuvers. The sensor(s) 212 may include, for example, one ormore lidar sensors, one or more cameras (e.g. RGB-cameras, intensity(grey scale) cameras, infrared cameras, depth cameras, stereo cameras),one or more magnetometers, one or more radar sensors, one or more sonarsensors, one or more microphones for sensing sounds, one or more IMUsensors (e.g., including accelerometers and gyroscopes), one or more GPSsensors, one or more Geiger counter sensors, one or more drive systemsensors, and/or other sensors related to the operation of the examplevehicle system 202. Other sensors may include, for example, a speedsensor, sensors related to operation of internal combustion engines,batteries, fuel cells, and/or electric motors, sensors related to thetires to detect tire temperature, tire pressure, and/or tread depth,and/or brake-related sensors for detecting brake temperatures and/orwear, and in vehicles having regenerative braking, sensors for detectingparameters related to operation of the regenerative braking system.

The example vehicle system 202 may include a vehicle controller 226configured to generate control signals for components of the vehiclesystem 202. For example, the vehicle controller 226 may receive a sensorsignal from the sensor(s) 212 and may be configured to generate acontrol signal for the network interface 210 to establish acommunications link and/or transmit a message to a teleoperations systemfor a teleoperations intervention, a control signal to activate a sensor212 and/or record sensor data, a control signal for the input/outputdevice(s) 214 to send or receive data from passenger(s) or surroundingsof the vehicle system 202, and/or a control signal for the drive system216 to control movement of the vehicle system 202. To generate a controlsignal, the vehicle controller 226 may include one or more ML modelshaving different functionalities, as subsequently discussed in moredetail. In some examples, the vehicle controller 226 may includehardware and/or software components. In some examples, the vehiclecontroller 226 may include instructions stored in the memory 206 that,when executed by processor(s) 204, cause the processor(s) 204 and/or thevehicle system 202 to conduct the operations discussed herein.

The example vehicle system 202 may also include a distributed processingmanager 228 that is configured to coordinate operations of multiplevehicles. In some examples, the distributed processing manager 228 mayinclude a virtual machine manager (e.g., a hypervisor), a containermanger, and/or a single system image manager for abstracting discreteresources across multiple vehicles and load-balancing processing andmemory requests across the multiple vehicles. In some examples, thedistributed processing manager 228 may relay or assign processing tasks,memory storage, processing results, vehicle maneuvers, vehicleassignments, etc. between multiple vehicles. In some examples, when avehicle and/or the remote computing system 222 detects a condition thatrequires an elevated number of resources, the distributed processingmanager 228 may coordinate operations of multiple vehicles that arerouted to the site requiring the elevated number of resources. Putsimply, the vehicles may be organized to provide mobile cloud computingand the distributed processing manager 228 may configure on-the-fly setup and management of distributed computing among multiple vehicles.

The example vehicle system 202 may also include at least one virtualmachine 230 (and/or a container), though any number of virtual machinesare contemplated. In some examples, conducting testing and/or trainingat the vehicle system 202 may include running the testing and/ortraining using an unmodified vehicle controller 226 (e.g., no additionalor alternate ML models, which is to say using all normative models),and/or a second instantiation of the vehicle controller run in thevirtual machine 230 or a separate container (i.e., using a separateruntime system for the second instantiation from the runtime system ofthe first controller, managed by a container manger), experimentalvehicle controller 232. The experimental vehicle controller 232 mayinclude additional and/or alternate ML models as compared to thenormative ML models of the vehicle controller 226. For example, theexperimental vehicle controller 232 may include a target ML model thatis identical to a corresponding ML model of the vehicle controller 226before the testing and/or training but that may be modified during thetesting and/or training, for example by training; and/or an experimentalmodel (i.e., an ML model that has a different architecture (e.g. anumber of layers, connections between nodes, layer types, etc.) than acorresponding existing ML model, which may include the same or differentinput(s)/output(s)).

In some examples, conducting testing and/or training at the vehiclesystem 202 may include receiving synthetic data (e.g., images, a seriesof GPS locations, pose data, a series of velocity and/or accelerationdata captured by a sensors other than the sensors of the vehicle and/orcreated by a simulation of a computing device) and conducting at leastpart of the testing and/or training using the synthetic data. Forexample, the vehicle system 202 may use synthetic data, (instead ofreal-time data gathered at the vehicle) such as previously capturedand/or generated lidar point cloud, video, images, etc. and/or may notsend an output control signal to components of the vehicle system 202 sothat the vehicle generates an output control signal, but does notactually actuate the associated components.

The virtual machine 230 (or container) and/or the vehicle controller 226may include a performance analyzer 234 that receives sensor data fromsensor(s) 212, output of an ML model of the vehicle controller (e.g.,confidence levels of one or more ML models of the vehicle controller226), and/or output control signals from the vehicle controller 226 todetermine performance metrics 236 of the operation of the vehicle system202. In some examples, the performance analyzer 234 may include temporallogic that scores operation of the drive system. In some examples, theperformance analyzer 234 may be configured to compare temporal logicscores, confidence levels, processor load of an ML model, amount ofenergy consumed, speed of achieving a result by an ML model, and/or anumber of teleoperator interventions during operation of the vehiclesystem 202 using a target and/or experimental ML model to operation ofthe vehicle system 202 using normative ML models. The performancemetrics 236 may include an indication that a target and/or experimentalML degrades or improves performance of the vehicle system 202. Modelshaving performance metrics outperforming current models may beconsidered as validated models. Validated models may be transmitted to aremote computing system for dissemination to a fleet of vehicles, or aportion thereof.

In some examples, the virtual machine 230 or a hypervisor of the virtualmachine (or a container manager) may record a machine state 238 that mayinclude a state of the experimental vehicle controller 232, a state ofthe performance analyzer 234, and/or a state of the machine, such thattesting and/or training being conducted at the vehicle system 202 may bemigrated to and/or analyzed at another device. In the case of machinelearning models, the machine state 238 may also include networkarchitectures, relative weights of connections, number of processediterations, etc. (where the model is a neural network). The machinestate 238 is a record sufficient to allow a second device (or devices)to resume operations of the virtual machine 230 (or container),experimental vehicle controller 232, and/or performance analyzer 234from a time at which the state was recorded. For example, the machinestate 238 may include copied memory pages, flag states, runtime states,operation system states, and/or application states. In some examples,the vehicle system 202 may transmit this information to another vehicle218 and/or a remote computing device 222 via the network interface 210.In additional or alternate examples, a machine state of the nativeoperating system may be saved (e.g., checkpointed), including, forexample, network architectures, relative weights of connections, numberof processed iterations, etc. (where the model is a neural network).

In some examples, the vehicle system 202 may be controlled by output ofeither the vehicle controller 226 or the experimental vehicle controller232. Both may be run simultaneously, although in some examples, thevehicle system 202 may run one at a time. If the performance metrics 236indicate that a target and/or experimental ML model degrades performanceof the vehicle system 202, the vehicle system 202 may terminatecontrolling the vehicle system 202 by the vehicle controller that isconfigured with the target and/or experimental ML model, complete thetesting and/or training using synthetic data and/or by producing asynthetic output (i.e., generating a control signal but not sending thecontrol signal to corresponding component(s)), and/or complete thetesting and/or training at a remote computing device. In examples usinga virtual machine or container, the vehicle system 202 may revert tocontrolling the vehicle system 202 using the vehicle controller 226instead of the experimental vehicle controller 232. If the performancemetrics 236 indicate that a target and/or experimental ML model improvesperformance of the vehicle system 202, the vehicle system 202 mayreplace and/or add the target and/or experimental ML model to thevehicle controller 226 as a normative ML model. This may be done withconfirmation from a remote computing device 222 received via the networkinterface 210, in some examples.

In some examples, the remote computing device 222 may includedistributed cloud computing devices and/or a teleoperations device. Theremote computing device 222 may provide fleet control for multiplevehicles. In some examples, the remote computing device 222 maydetermine that one or more vehicles are available to conduct testingand/or training and, based on that determination, may transmitinstructions (i.e., a “test mission”) to the one or more vehicles toconfigure them to conduct the testing and/or training, which may includean experimental ML model. Additionally, or alternatively, the vehiclesystem 202 may send an indication to the remote computing device 222that it has available processing bandwidth to perform computationsand/or testing and/or training. In some examples, the remote computingdevice 222 may receive a trained target ML model from the one or morevehicles, performance metrics 236, a machine state 238, and/or recordedsensor data. The remote computing device 222 may use this data todetermine whether performance of the vehicle system 202 was improved ordegraded by inclusion of the target and/or experimental ML model. If theremote computing device 222 determines that a model improved performanceof the vehicle system 202 (i.e., the remote computing device 222“validates” the model), the remote computing device 222 may transmit thevalidated model to one or more vehicles of a fleet of vehicles. In someexamples, the remote computing device 222 may transmit instructions toexchange one or more existing ML models of the one or more vehicles withthe validated model. Additionally, or alternatively, the remotecomputing device 222 may instruct the one or more vehicles to use thevalidated model in addition to or instead of an existing ML model when avehicle is in a particular location or experiencing a particularcondition. In some examples, the remote computing device 222 maytransmit the validated model to a vehicle if the remote computing device222 and/or the vehicle detects that the vehicle is about to encounter alocation and/or condition associated with the validated model.

FIG. 3 includes a block diagram of an example vehicle controller 300 andan example experimental vehicle controller 302. Example vehiclecontroller 300 may represent vehicle controller 226 and exampleexperimental vehicle controller 302 may represent experimental vehiclecontroller 232. Example vehicle controller 300 and/or exampleexperimental vehicle controller 302 may include ML models configured toperform various operations such as, for example, localizing theautonomous vehicle, detecting objects, classifying objects, trackingobjects, generating trajectories, selecting one or more trajectories forcontrolling the autonomous vehicle, determining whether to communicatedata with a teleoperations device, etc. For example, example vehiclecontroller 300 and/or example experimental vehicle controller 302 mayinclude a perception engine 304 that includes an object detector 306,object classifier 308, an object tracker 310, etc.; a teleoperationsintermediary 312 for determining to transmit a message to a remotecomputing device to receive assistance at the vehicle; a localizer 314for determining a pose of the vehicle; a planner 316 for determining aroute from a first location to a second location, generating potentialtrajectories for controlling motion of the vehicle, and selecting one ormore trajectories as a current and/or contingent trajectory of thevehicle; a motion controller 318 for generating a drive control signalfrom a selected trajectory and known environmental/vehicle data (e.g.,tire slip likelihood, etc.); etc.

Any of these models may use sensor data 320 received from sensor(s) 212and/or outputs of other models as input and may generate outputs thatare used, ultimately, to generate a control signal 322 for controllingvarious components of the vehicle system 202 (e.g., network interface210, sensor(s) 212, input/output device(s) 214, drive system 216).

Vehicle controller 300 includes an illustration of a set of “normative”ML models that may be used to control the vehicle system 202 duringnormal operation of the vehicle. The experimental vehicle controller 302may include an ML model that is substituted for a normative ML model,such as the experimental object tracker 324 that is depicted in place ofobject tracker 310. Experimental object tracker 324 and object tracker310 may have a same function to receive sensor data 320 and/or outputsof other ML models and to output a track (e.g., historical, current,and/or predicted heading, position, velocity, and/or acceleration) foran object that has been detected by the perception engine. However, theexperimental object tracker 324 and object tracker 310 may differ inthat they may be ML models of a different type (e.g., one may be aconvolutional neural network whereas the other is a directed acyclicgraph) and/or, at the very least, the two models may have a differentarchitecture (e.g. a number of layers, connections between nodes, layertypes, etc.). For example, even if they are of the same type, nodes andconnections between the nodes may differ.

FIG. 4A depicts an example testing and/or training scenario 400involving multiple vehicles 104(1)-104(3). In some examples, a remotecomputing device (or a computing device onboard one of the vehicles104(1)-(3)) may determine that diverting one or more vehicles will beunlikely to cause fleet performance indicators to meet a threshold(i.e., the fleet performance would be unlikely to degrade by the one ormore vehicles not being “in-service” during the testing and/ortraining). In some examples, a remote computing device may transmit atest mission to multiple vehicles 104(1)-104(3) that includes alocation, a condition, and/or instructions for at least one maneuverand/or mode for one or more of the vehicles. For a test mission thatincludes multiple autonomous vehicles, each vehicle may operateaccording to a same or different mode. The vehicles may be connected viaan ad hoc network 402 and may coordinate assignment of modes via the adhoc network 402 and distributed processing managers stored at therespective vehicles.

In some examples, a first vehicle 104(1) may operate according to asubject mode as a “subject vehicle” where the first vehicle 104(1) has atarget ML model and/or experimental ML model stored thereon and trainsand/or tests the target and/or experimental ML model. A second vehicle104(2) may operate according to an observation mode where the secondvehicle 104(2) captures sensor data that is used to develop a targetand/or experimental ML model and/or to evaluate operation of the subjectvehicle (e.g., this may include positioning the vehicle where it canrecord the subject vehicle from a “third-party” perspective). A thirdvehicle 104(3) may operate according to a scenario mode where theautonomous vehicle operates to set up a scenario for the benefit of thesubject vehicle (e.g., driving erratically, flickeringheadlights/taillights, blocking a portion of the roadway, etc. to assistin training and/or testing operation of the subject vehicle using thetarget and/or experimental ML model). Example scenario 400 depicts thethird vehicle 104(3) performing an erratic driving or swerving maneuverfor the benefit of training or testing an ML model at the first vehicle104(1). The second vehicle 104(2) operates in an observation mode inexample scenario 400 to provide additional sensor data and/or processingand/or storage capacity for testing and/or training the ML model at thefirst vehicle 104(1). In some examples, the second vehicle 104(2) and/orthe third vehicle 104(3) may also operate in a subject mode to testand/or train a same or different ML model at the respective vehicles.

FIG. 4B depicts an example coordination between two vehicles, 104(1) and104(2) to train and/or test an ML model, according to a variety ofexamples. In some examples, the first vehicle 104(1) may migrate testingand/or training to second vehicle 104(2) by transmitting a machine state404 (including network architectures, current weights of connections,numbers of iterations, training data, etc.), any performance metrics 406that may have been determined by the first vehicle 104(1), an updatedmodel 408 (which may include a trained target ML model) and/or anexperimental model 410 to the second vehicle 104(2). This data may beincomplete.

In some examples, distributed processing managers of the respectivevehicles may transmit coordination instructions 412 between each otherto coordinate sensor data capture, assigning testing and/or trainingmodes (e.g., subject, observer, scenario), distributing processingand/or memory loads among the vehicles, etc.

In some examples, the vehicles, 104(1) and 104(2), may act as mobilecloud computing devices that lend mobile sensing, processing, memory,and/or I/O to another vehicle and/or according to a test mission from aremote computing device. For example, the remote computing device mayinstruct one or more vehicles to provide mobile cloud computing serviceat or near a location so that a network is not burdened by transmittingdata to the remote computing device, additional sensors and/or memoriesare available, and/or processing capacity is increased for monumentalprocessing tasks (e.g., on-the-fly deep learning, convolutional signalfiltration). In some examples, mobile cloud computing may additionallyinclude edge computing, which may include conducting computations,storage, network functions, etc. at an edge device such as, for example,Internet-of-Things (IoT) devices, integrated access devices (IADs),multiplexers, metropolitan area network (MAN) devices, wide area network(WAN) devices, a computing device of a vehicle, and/or other devicesthat provide an entry point to an enterprise or service provider.

To provide mobile cloud computing capabilities and/or to lend computingresources (e.g., sensor(s), processor(s), memory, I/O capabilities) totesting and/or training, vehicles 104(1) and 104(2) may form a singlesystem image across the resources of the vehicles 104(1) and 104(2),managed by a one or more distributed processing managers of the vehicles104(1) and 104(2). The vehicles 104(1) and 104(2) may therefore relaysingle system image instructions 414 between the vehicles according tothis example. In an additional or alternative example to using a singlesystem image, a distributed processing manager of one of the vehiclesmay load balance processing requests and transmit parallel processinginstructions 416 to the other vehicle to coordinate parallel processingdata between the two vehicles.

FIG. 5 illustrates a flow diagram of an example process 500 forcontrolling a vehicle to divert from an in-service state to conducttesting and/or training.

At operation 502, the example process 500 may include receiving anindication that a vehicle has unallocated computational resources,according to any of the techniques discussed herein. In some examples,the vehicle may receive the indication via a network interface from aremote computing device and/or the vehicle may receive the indicationfrom a determination conducted by a processor of the vehicle. In someexamples, a computing system onboard the vehicle may make adetermination that the vehicle is capable of performing additionalcomputations and or testing and/or training. In some examples, a testmission may accompany the indication. In some examples, this may includemaking a determination that transitioning the vehicle from an in-servicestate to an out-of-service state will not impact fleetwide service suchthat a threshold of fleetwide service is met.

At operation 504, the example process 500 may optionally includetransitioning the vehicle from an in-service state to an out-of-servicestate, based at least in part on determining that the vehicle hasunallocated computational resources, according to any of the techniquesdiscussed herein. In some examples, the vehicle may transition from anin-service state to an out-of-service state at a time or upon meeting acondition specified by a test mission. In some examples, a vehicle in anin-service state responds to requests to complete tasks (e.g., requestsfor a ride, requests to complete an excavation) whereas in anout-of-service state the vehicle does not respond to these requests. Insome examples, the vehicle may transition back to an in-service statefrom an out-of-service state if a current fleet performance metricindicates that a threshold is met or is likely to be met (e.g.,estimated average arrival time of a vehicle is likely to increase to atleast meet a threshold average arrival time in the next half hour), ifthe testing and/or training is otherwise frustrated (e.g., performancemetrics indicate that an ML model performs poorly enough to terminatethe testing and/or training, impossibility of completion), a time periodfor completing the testing and/or training expires, etc. In someexamples, operation 504 may be bypassed. In such examples, additionalcomputation and/or testing and/or training may be performed while thevehicle remains “in-service” by using spare processor cycles (e.g.,processor cycles that are not allocated to a thread).

At operation 506, the example process 500 may include receiving acommand to utilize the unallocated computational resources. This may bedone while the vehicle is in the in-service state, in some examples, orin the out-of-service state, in other examples. In some examples, thecommand may include a command to conduct testing and/or training orperforming the additional computation at the vehicle. The trainingand/or testing may be accomplished by (506(A)) operating the vehicleusing an experimental ML model and/or (506(B)) training a target MLmodel of the vehicle controller to obtain an updated ML model based atleast in part on operating the vehicle using the target ML model,according to any of the techniques discussed herein. In some examples,the vehicle may conduct 506(A) and/or 506(B) without transitioning to anout-of-service state.

FIG. 6 illustrates a flow diagram of an example process 600 forconducting testing and/or training to train or test an ML model at avehicle.

At operation 602, the example process 600 may include receiving anindication that the vehicle has or will have unallocated computationalresources at a time period, according to any of the techniques discussedherein. In some examples, the indication is an indication that thevehicle has been selected. The vehicle may be selected to conducttraining, testing, and/or additional computations based at least in parton a current or anticipated location of and/or condition at the vehicle.For example, selecting the vehicle based on the anticipated locationand/or condition may allow the vehicle to test an ML model for an edgecase (e.g., a situation that does not frequently arise) known to occurin that location and/or condition. In some examples, the indication maybe accompanied by a test mission that includes instructions forconducting testing and/or training. For example, the test mission mayinclude a time, location (e.g., a safe location for testing and/ortraining an ML model), condition, maneuver, and/or manner ofestablishing an experimental control (e.g., identifying and/ortransmitting a set of data to use as the experimental control and/orinstructing the vehicle to perform experimental control maneuvers).

At operation 604, the example process 600 may include causing thevehicle controller to train a target ML model of the vehicle, creating atrained model; and/or test an experimental ML model, according to any ofthe techniques discussed herein.

At operation 606, the example process 600 may include controlling thevehicle according to an output of the trained model and/or test theexperimental ML model, based at least in part on receiving theinstructions and at the beginning of the time period, according to anyof the techniques discussed herein.

FIG. 7 illustrates a flow diagram of an example process 700 forutilizing spare computational resources of a vehicle to train and/ortest an ML model. In some examples, the example process 700 may beconducted on a remote computing device.

At operation 702, the example process 700 may include determining that avehicle has unallocated computational resources during a time period,according to any of the techniques discussed herein. In some examples,this may include accessing historic fleet data and determining alikelihood that the vehicle will be available during the time period. Insome examples, a computing system onboard the vehicle may determine anexcess computing ability and provide an indication of the excesscomputing ability to the remote computing system.

At operation 704, the example process 700 may include transmitting tothe vehicle instructions to conduct a test mission during the timeperiod and one or more of an experimental ML model or instructions totrain a target ML model stored at the vehicle, according to any of thetechniques discussed herein. The remote computing device mayadditionally or alternatively determine and/or receive a location,condition, maneuver, and/or manner of establishing an experimentalcontrol to include in the instructions to conduct a test mission at thevehicle.

At operation 706, the example process 700 may include receiving one ormore of sensor data, an updated target ML model, or performance metricsfrom the vehicle that correspond to completion of at least a portion ofthe test mission, according to any of the techniques discussed herein.

At operation 708, the example process 700 may include determining thatone or more of the updated target ML model or the experimental ML modelchanged performance of the vehicle (e.g., improved, maintained, ordegraded performance of the vehicle), according to any of the techniquesdiscussed herein. In some examples, the example process 700 may furtherinclude transmitting the updated target ML model and/or the experimentalML model to one or more vehicles of a fleet if the model improves ormaintains performance of the vehicle. This transmission may beaccompanied with instructions to replace a normative ML model with theupdated target ML model and/or the experimental ML model and/orinstructions to add the updated target ML model and/or the experimentalML model to an ML pipeline.

Example Clauses

A. A vehicle comprising: a vehicle controller operably connected to atleast one sensor, a network interface, and a drive system to controloperation of the vehicle, the vehicle controller being configured to:receive an indication that the vehicle controller has unallocatedcomputational resources; receive a command to utilize the unallocatedcomputational resources, the command causing the vehicle controller tobe further configured to conduct one or more of: testing an experimentalmachine learning (ML) model based at least in part on causing the drivesystem to control operation of the vehicle using the experimental MLmodel, the testing yielding a test result, or training a target ML modelto create a trained ML model, based at least in part on causing thedrive system to control operation of the vehicle using the target MLmodel.

B. The vehicle of paragraph A, wherein the indication that the vehiclecontroller has unallocated computational resources comprises one or moreof: an indication that a service demand for the vehicle is unlikely tomeet a threshold, based at least in part on present or historical fleetinformation, an indication that the vehicle is idle, or an indicationthat a service demand for the vehicle is unlikely to meet a thresholdduring a time period.

C. The vehicle of paragraph B, wherein the threshold is one or more of anumber of requests for rides, an average wait time before arrival of avehicle, or a ratio of vehicles in-service to requests.

D. The vehicle of either paragraph B or C, the vehicle controller beingfurther configured to terminate conducting the testing or training basedat least in part on one or more of: determining that the time period haslapsed, receiving a task for the vehicle to perform, or determining orreceiving an indication that the service demand for the vehicle hasincreased past the threshold.

E. The vehicle of any one of claims A-D, wherein: the vehicle controlleris configured to conduct an evaluation, and testing the experimental MLmodel further comprises: determining instructions for controlling theautonomous vehicle from the vehicle controller based at least in part onan output of the experimental ML model; and determining performancemetrics of the experimental ML model based at least in part on controlof the vehicle according to the instructions.

F. The vehicle of any one of claims A-E, wherein: the vehicle controlleris a first instance of the vehicle controller that comprises a currentML model, and conducting one or more of the testing or the trainingcomprises: instantiating a second vehicle controller in a virtualmachine or container, the second vehicle controller comprising one ormore of the experimental ML model or the target ML model; receivingfirst instructions for controlling the vehicle from the first instanceof the vehicle controller at a first time; receiving second instructionsfor controlling the vehicle from the second instance of the vehiclecontroller at a second time, the second time being substantially thesame as the first time; controlling the vehicle according to the firstinstructions or the second instructions; and determining performancemetrics of one or more of the first instance of the vehicle controlleror the second vehicle controller based at least in part on one or moreof the first instructions, the second instructions, or sensor data.

G. The vehicle of any one of claims A-F, wherein: the vehicle receives atest mission via the network interface, the test mission including alocation and a condition; and the vehicle controller is configured toconduct one or more of testing or training, the testing and trainingconfigured to consume sensor data associated with the location and thecondition.

H. The vehicle of paragraph G, wherein one or more of the location orthe condition correlates with a decrease in vehicle performance, andfurther wherein the sensor data is previously recorded sensor data.

I. The vehicle of any one of claims A-H, wherein conducting one or moreof the testing or the training includes: instantiating a virtual machineand/or container on a processor of the vehicle; and evaluating one ormore of the experimental ML model or the target ML model on the virtualmachine and/or container.

J. The vehicle of paragraph I, the vehicle controller being furtherconfigured to: terminate conducting the testing or the training; andmigrate, via the network interface of the vehicle, one or more of amachine state, machine state, sensor data, performance metrics, theexperimental ML model, or the trained ML model to one or more of aremote computing device or another vehicle, based at least in part onthe termination.

L. The vehicle of paragraph K, wherein one or more of the machine state,sensor data, performance metrics, the experimental ML model, or theupdated ML model are configured to enable the one or more of the remotecomputing device or the other vehicle to resume the evaluation from apoint at which the evaluation was terminated.

M. The vehicle of any one of claims A-L, the vehicle controller beingfurther configured to: form an ad hoc connection with one or more othervehicles via the network interface; receive a test mission from a remotecomputing device via the network interface; and transmit instructions tothe one or more other vehicles to cause the one or more other vehiclesto one or more of conduct a maneuver or change a mode of operation,according to the test mission.

N. The vehicle of any one of claims A-M, the vehicle controller beingfurther configured to: form an ad hoc connection with one or more othervehicles via the network interface; transmit a portion of data to beprocessed for the evaluation to at least one of the one or more of thevehicles; and receive a processing result determined by the at least oneof the one or more of the vehicles.

O. The vehicle of paragraph N, wherein forming the ad hoc connectionincludes instantiating a single system image across the one or morevehicles or instantiating a virtual machine cluster and/or containercluster among the one or more vehicles.

P: The vehicle of any one of claims A-O, wherein the vehicle controlleris further configured to transition the vehicle from an in-service stateto an out-of-service state, based at least in part on determining thatthe vehicle has unallocated computational resources.

Q. A method of controlling a vehicle comprising: receiving an indicationthat a vehicle controller of the vehicle has or will have unallocatedcomputational resources at a time period; based at least in part onreceiving the indication, causing the vehicle controller to: train atarget machine-learning model, creating a trained model, or test anexperimental model; and control the vehicle according to an output ofthe trained model or the experimental model.

R. The computer-implemented method of paragraph Q, wherein the methodfurther comprises receiving a test mission, the test mission comprisingone or more of a location or condition; and the vehicle controller isfurther configured to evaluate the experimental model or the trainedmodel based at least in part on the test mission.

S. The computer-implemented method of paragraph R further comprising:instantiating a virtual machine and/or container to determine an outputof the trained model or the experimental model; determining that thevehicles is not able to complete the test mission; terminating trainingthe trained model or testing the experimental model according to thetest mission; recording one or more of a machine state, a machine state,sensor data, performance metrics, the experimental model, or an updatedmodel derived from training the target model; and at least one of:transmitting one or more of the machine state, sensor data, performancemetrics, the experimental model, or the updated model to one or more ofa remote computing device or another vehicle; or resuming the testmission using one or more of the machine state, the machine state, thesensor data, the performance metrics, the experimental model, or theupdated model, based at least in part on receiving an indication thatthe excess computing resources are subsequently available.

T. A non-transitory computer-readable medium having a set ofinstructions that, when executed, cause one or more processors toperform operations comprising: determining that a vehicle hasunallocated computational resources during a time period; transmittingto the vehicle instructions to conduct a test mission during the timeperiod and one or more of an experimental model or instructions to traina target model stored at the vehicle; receiving one or more of sensordata, an updated target model, or performance metrics from the vehiclethat correspond to completion of at least a portion of the test mission;and determining that one or more of the updated target model or theexperimental model changed performance of the vehicle.

U. The non-transitory computer-readable medium of paragraph T, the setof instructions further causing the one or more processors to performoperations comprising transmitting one or more of the trained model orthe experimental model to one or more additional vehicles of a fleet,based at least in part on determining that one or more of the trainedmodel or the experimental model improves performance of the vehicle.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described. Rather,the specific features and acts are disclosed as example forms ofimplementing the claims.

The modules described herein represent instructions that can be storedin any type of computer-readable medium and can be implemented insoftware and/or hardware. All of the methods and processes describedabove can be embodied in, and fully automated via, software code modulesand/or computer-executable instructions executed by one or morecomputers or processors, hardware, or some combination thereof. Some orall of the methods can alternatively be embodied in specialized computerhardware.

Conditional language such as, among others, “can,” “could,” “may” or“might,” unless specifically stated otherwise, are understood within thecontext to present that certain examples include, while other examplesdo not include, certain features, elements and/or steps. Thus, suchconditional language is not generally intended to imply that certainfeatures, elements and/or steps are in any way required for one or moreexamples or that one or more examples necessarily include logic fordeciding, with or without user input or prompting, whether certainfeatures, elements and/or steps are included or are to be performed inany particular example.

Conjunctive language such as the phrase “at least one of X, Y or Z,”unless specifically stated otherwise, is to be understood to presentthat an item, term, etc. can be either X, Y, or Z, or any combinationthereof, including multiples of each element. Unless explicitlydescribed as singular, “a” means singular and plural.

Any routine descriptions, elements or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode that include one or more computer-executable instructions forimplementing specific logical functions or elements in the routine.Alternate implementations are included within the scope of the examplesdescribed herein in which elements or functions can be deleted, orexecuted out of order from that shown or discussed, includingsubstantially synchronously, in reverse order, with additionaloperations, or omitting operations, depending on the functionalityinvolved as would be understood by those skilled in the art.

It should be emphasized that many variations and modifications can bemade to the above-described examples, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

1. (canceled)
 2. A vehicle comprising: one or more processors; andmemory storing instructions that, when executed by the one or moreprocessors, cause the vehicle to perform operations comprising:determining an amount of unused computational resources associated withthe vehicle, the amount of unused computational resources comprising atleast one of an amount of free memory of the memory or a percentage ofunused processing of the one or more processors; and processing, as acomputational task and based at least in part on the amount of unusedcomputational resources, at least one of: testing a machine learnedmodel, or training a machine learning model to create a trained machinelearned model.
 3. The vehicle of claim 2, wherein the computational taskis processed while the vehicle is operated in an in-service state or anout-of-service state.
 4. The vehicle of claim 2, wherein the vehicle isone of a fleet of vehicles, and wherein processing the computationaltask is further based at least in part on a service demand associatedwith the fleet of vehicles being less than or equal to a thresholddemand.
 5. The vehicle of claim 2, wherein testing the machine learnedmodel comprises controlling the vehicle based at least in part on anoutput of the machine learned model and determining a result.
 6. Thevehicle of claim 5, wherein the operations further comprise transmittingthe machine learned model to at least one of an additional vehicle or aserver based at least in part on the result.
 7. The vehicle of claim 2,wherein determining the amount of unused computational resources isbased at least in part on at least one of a current route of the vehicleor availability of at least one fleet vehicle from historical data. 8.The vehicle of claim 2, wherein the computational task is performeduntil a time period has lapsed, a task for the vehicle to perform isreceived, or a service demand for the vehicle has increased passed athreshold.
 9. The vehicle of claim 8, wherein the threshold is based atleast in part on a number of requests for rides, an average wait time,or a ratio of a number of vehicles to a number of requests.
 10. A methodof controlling a vehicle comprising: determining an amount ofunallocated computational resources, computational resources comprisingone or more memory or processor operations of a computing system of avehicle; and processing, based at least in part on the amount ofunallocated resources, a computational task, the computational taskcomprising at least one of: testing a machine learned model, or traininga machine learning model to create a trained machine learned model. 11.The method of claim 10, wherein the machine learned model or the trainedmachine learned model is used to identify an object in an environmentsurrounding the vehicle, plan a route for the vehicle, plan a trajectoryfor the vehicle, or interact with an object, passenger, or human. 12.The method of claim 10, wherein the other machine learned model is usedto handle previously unencountered scenarios or to improve one or moredriving parameters for the vehicle.
 13. The method of claim 10, whereinprocessing the computational task further comprises: receiving a testmission comprising a location or condition; and processing thecomputational task is based at least in part on the test mission. 14.The method of claim 10, wherein testing the machine learned modelcomprises controlling the vehicle according to an output of the machinelearned model and determining a result.
 15. The method of claim 10,wherein the vehicle is one of a fleet of vehicles, and whereinprocessing the computational task is further based at least in part onan availability of fleet vehicles from historical data.
 16. One or morenon-transitory computer-readable media having a set of instructionsthat, when executed, cause one or more processors to perform operationscomprising: determining an amount of unallocated computational resourcesassociated with a computing system of a vehicle; and based at least inpart on the amount of unallocated computational resources, at least oneof: testing a machine learned model, or training a machine learningmodel to create a trained machine learned model.
 17. The one or morenon-transitory computer-readable media of claim 16, wherein testing themachine learned model comprises controlling the vehicle based at leastin part on an output of the machine learned model and determining aresult.
 18. The one or more non-transitory computer-readable media ofclaim 17, wherein the operations further comprise transmitting, based atleast in part on the result, the machine learned model to at least oneof an additional vehicle or a server accessible by the additionalvehicle.
 19. The one or more non-transitory computer-readable media ofclaim 16, wherein the at least one of testing the machine learned modelor training the machine learning model is further based at least in parton a state of the vehicle.
 20. The one or more non-transitorycomputer-readable media of claim 16, wherein the amount of unallocatedcomputational resources comprises at least one of an amount of unusedmemory or an amount of unused processing operations, and wherein the atleast one of testing the machine learned model or training the machinelearning model is performed until at least one of a task for the vehicleto perform is received or until a service demand for the vehicle meetsor exceeds a threshold.
 21. The one or more non-transitorycomputer-readable media of claim 16, wherein the vehicle is one of afleet of vehicles, and wherein the at least one of testing the machinelearned model or training the machine learning model is further based atleast in part on a utilization of the fleet.